Zeroth and First Order Stochastic Frank-Wolfe Algorithms for Constrained Optimization

This paper considers stochastic convex optimization problems with two sets of constraints: (a) deterministic constraints on the domain variable, which are difficult to project onto; and (b) or that admit efficient projection. Problems this form arise frequently in context semidefinite programming as well when various NP-hard solved approximately via relaxation. Since projection onto first set is difficult, it becomes necessary explore projection-free algorithms, such Frank-Wolfe (FW) algorithm. On other hand, second cannot be handled same way, must incorporated an indicator function within objective function, thereby complicating application FW methods. Similar have been studied before; however, they suffer from slow convergence rates. work, equipped momentum based gradient tracking technique, guarantees fast rates par best-known for without constraints. Zeroth-order variants proposed algorithms also developed again improve upon state-of-the-art rate results. We further propose novel trimmed enjoy their classical counterparts, but empirically shown require significantly fewer calls linear minimization oracle speeding up overall The efficacy tested relevant applications sparse matrix estimation, clustering relaxation, uniform sparsest cut problem.